Synoptic: Summarizing System Logs with Refinement

Distributed systems are often difficult to debug and understand. A typical way of gaining insight into system behavior is by inspecting execution logs. However, manual inspection of logs is an arduous process. To support this task we developed Synoptic. Synoptic outputs a concise graph representation of logged events that captures temporal invariants mined from the log. We applied Synoptic to synthetic and real distributed system logs and found that it augmented a distributed system designer’s understanding of system behavior with reasonable overhead for an offline analysis tool. In contrast to prior approaches, Synoptic uses a combination of refinement and coarsening to explore the space of representations. Additionally, it infers temporal event invariants to capture distributed system semantics. These invariants drive the exploration process and are satisfied by the final representation.